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PREFACE 


This  effort  was  conducted  under  Work  Unit  1 123-B1-01,  Pilot  Selection  and 
Classification  Support,  which  is  dedicated  to  research  into  the  selection  and  classification 
of  U.S.  Air  Force  aircrew  personnel.  Send  written  correspondence  to  AFRL/HEAB,  7909 
Lindbergh  Drive,  Brooks  AFB,  TX  78235-5352.  Send  electronic  mail  to 
carretta@al  h  rm .  brooks.af.m  i  I . 


TRANSFER  OF  TRAINING  EFFECTIVENESS 
IN  FLIGHT  SIMULATION:  1986  TO  1997 


INTRODUCTION 

Over  the  past  several  decades,  researchers  have  sought  to  determine  the  effectiveness  of 
flight  simulators  as  augmentation  for  “hands-on”  flying  training.  Simulators  have  been  proposed 
for  use  to  save  money,  extend  the  life  of  the  aircraft,  maintain  proficiency  when  not  performing 
“hands-on”  flying,  and  provide  more  effective  training  especially  in  areas  difficult  to  train  in 
operational  aircraft  (e.g.,  emergency  procedures).  The  purpose  of  this  report  is  to  review  and 
summarize  the  literature  from  1986  to  1997  regarding  the  transfer  of  training  from  the  simulator 
to  the  aircraft.  Sixty-seven  articles,  conference  papers,  and  technical  reports  regarding  simulator 
flying  training  and  transfer  were  identified.  Of  these,  only  13  were  related  directly  to  transfer  of 
training.  For  the  interested  reader,  the  Appendix  contains  references  regarding  the  54  flying 
training  studies  that  were  not  directly  related  to  transfer  of  training. 


TRANSFER  OF  TRAINING  STUDIES 

Early  Studies:  1957  to  1986 

Hays,  Jacobs,  Prince,  and  Salas  (1992)  reviewed  the  pilot  training  literature  from  1957  to 
1986.  Their  initial  survey  included  247  journal  articles,  book  chapters,  and  technical  reports 
regarding  training  effectiveness.  Of  these,  only  26  studies  included  information  sufficient  to  be 
included  in  a  meta-analysis  regarding  transfer  of  training  in  a  flight  simulator  to  operational 
aircraft  equipment.  Of  the  26  studies,  19  involved  jet  aircraft  and  7  involved  helicopters.  The 
meta-analysis  revealed  severed  trends  that  were  not  readily  apparent  from  the  individual  studies. 
First,  simulators  consistently  led  to  improved  training  effectiveness  for  jet  pilots  relative  to 
training  in  the  aircraft  only.  However,  this  was  not  true  for  helicopter  pilots.  Second,  motion 
cueing  was  not  found  to  add  to  training  effectiveness  for  jet  pilots  and,  in  some  instances,  may 
have  led  to  less-effective  training.  Motion  cueing  was  not  examined  for  helicopter  pilots  due  to 
the  small  number  of  studies.  Finally,  training  effectiveness  was  strongly  influenced  by  the 
amount  and  type  of  training  given  and  by  the  type  of  task  trained.  For  instance,  simulators  were 
more  effective  for  training  takeoff,  approach  to  landing,  and  landing  than  they  were  for  the 
combination  of  all  pilot  tasks. 

Recent  Studies:  1987  to  1997 

Since  the  mid-1980s,  several  studies  have  examined  transfer  and/or  quasi-transfer  of 
training  of  flying  skills  (Lintem  &  Garrison,  1992;  Lintem,  Roscoe,  Koonce,  &  Segal,  1990; 
Lintem,  Roscoe,  &  Sivier,  1990;  Lintem,  Sheppard,  Parker,  Yates,  &  Nolan,  1989;  Lintem, 
Taylor,  Koonce,  Kaiser,  &  Morrison,  1997;  Lintem,  Thomley- Yates,  Nelson,  &  Roscoe,  1987; 
Pfeiffer,  Horey,  &  Butrimas,  1991;  Taylor,  Lintem,  &  Koonce,  1993;  Westra,  et  al.,  1986; 
Wightman  &  Sistrunk,  1987).  Although  simulators  have  been  used  to  train  several  aspects  of 
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flying  (e.g.,  takeoff,  landing,  instruments,  control),  by  far,  the  most  frequently  investigated  use  of 
simulators  was  for  training  landing  skills. 

Landing  Skills.  Consistent  with  meta-analytic  results  found  in  Hays  et  al.  (1992),  several 
subsequent  studies  have  shown  simulators  to  be  effective  for  training  landing  skills.  Westra  et  al. 
(1986)  examined  the  effects  of  simulator  design  features  on  the  training  effectiveness  of  the  skill 
required  for  landing  on  aircraft  carriers.  They  compared  night  (i.e.,  low  detail)  versus  daytime 
(i.e.,  high  detail)  scenes  crossed  with  wide  versus  narrow  field  of  view.  Pilots  received  20,  40,  or 
60  simulator  trials.  Results  indicated  that  neither  scene  detail  nor  field  of  view  influenced 
effectiveness  of  transfer  from  the  simulator  to  the  aircraft.  However,  pilots  receiving  40  or  60 
simulator  trials  exhibited  better  landing  skills  than  those  who  received  only  20  trials. 

Lintem  and  his  colleagues  have  conducted  several  studies  investigating  the  effects  of  the 
amount  of  simulator  time,  scene  detail,  crosswind,  field  of  view,  display  orientation,  augmented 
feedback,  and  bank  control.  For  example,  Lintem,  Roscoe,  Koonce,  and  Segal  (1990)  examined 
transfer  of  landing  skills  from  a  flight  simulator  to  an  aircraft  in  early  flight  training.  One  group 
of  beginning  flight  students  was  given  two  sessions  of  practice  of  landing  skills  in  a  simulator 
before  starting  landing  practice  in  the  aircraft.  A  control  group  was  given  no  practice  in  the 
simulator  prior  to  practice  in  the  aircraft.  Results  indicated  that  the  experimental  group  required 
1.5  fewer  pre-solo  flying  hours  than  the  control  group.  This  reduction  in  flying  hours,  whether 
military  or  civilian,  could  lead  to  considerable  savings. 

Several  follow-on  studies  were  conducted  to  examine  the  effects  of  different  simulator 
attributes  on  the  transfer  of  landing  skills.  Lintem,  Roscoe,  and  Sivier  (1990)  investigated  effect 
of  scene  detail  (pictorial  vs.  symbolic),  display  orientation  (outside  in  or  inside  out),  bank  control 
order  (first  order  or  zero  order),  crosswind  (present  or  absent),  command  guidance  (constant, 
adaptive,  or  no  augmentation),  and  flight  path  prediction  (constant,  adaptive,  or  no  augmentation) 
on  transfer  of  landing  skill  in  the  simulator.  The  final  criterion  was  a  “conventional  inside  out 
pictorial  contact  display,  normal  simulator  control  dynamics,  and  a  5-knot  crosswind”  (p.  299). 
They  found  that  pictorial  displays  were  more  effective  than  symbolic  displays  and  normal  bank 
control  order  was  better  than  reduced  control  for  producing  transfer. 

Lintem  and  Garrison  (1992)  examined  transfer  effects  of  scene  detail  (high  pictorial,  low 
pictorial,  or  symbolic)  and  level  of  crosswind  (zero,  moderate,  or  high)  on  landing  performance 
in  a  simulator.  Participants  were  assigned  to  one  of  nine  training  conditions.  The  final 
performance  criterion  consisted  of  a  high  pictorial,  moderate  crosswind,  simulation.  Neither 
scene  detail  (high  vs.  low  pictorial)  nor  level  of  crosswind  had  a  significant  effect  on  transfer. 
Consistent  with  Lintem,  Roscoe,  and  Sivier  (1990),  pictorial  displays  led  to  better  transfer  than 
symbolic  displays. 

Lintem  et  al.  (1997)  investigated  effects  of  scene  detail  (low  or  moderate),  visual 
augmented  guidance  (off,  constant,  or  adaptive),  and  number  of  landing  training  trials  (0,  24,  48, 
or  72)  in  the  simulator  on  transfer  to  the  aircraft.  Those  in  the  zero  landing  trials  group  received 
instrument  training,  but  no  visual  display.  The  performance  criterion  was  the  number  of 
attempted  landings  prior  to  solo  in  the  operational  aircraft.  They  reported  that  students  who 


2 


trained  using  low  detail  had  better  transfer  compared  to  those  using  moderate  detail.  Further, 
augmented  guidance  enhanced  transfer  for  low  detail  scenes  but  degraded  transfer  for  high  detail 
scenes. 


In  summary,  simulators  were  shown  to  be  useful  for  training  landing  skills.  As  the 
number  of  simulated  sorties  increased,  performance  increased,  but  this  gain  appeared  to  level  off 
after  approximately  25  missions.  Field  of  view,  as  well  as  the  presence  or  absence  of  crosswinds, 
had  no  significant  effect  on  transfer.  The  results  concerning  scene  detail  are  unclear.  In  some 
instances,  greater  detail  led  to  better  performance,  while  in  other  instances,  it  did  not. 

Radial  Bombins  Accuracy.  While  a  majority  of  the  research  has  centered  on  landing  skill 
acquisition,  a  few  studies  have  been  conducted  to  determine  the  feasibility  of  training  bombing 
missions  in  the  simulator.  For  example,  Lintem  et  al.  (1987)  manipulated  scene  content  and 
augmented  feedback  in  a  quasi-transfer  training  study  of  air-to-ground  attack  skills.  Participants 
received  training  using  one  of  three  scene  detail  levels.  Final  performance  was  based  on 
simulated  bombing  missions  in  the  simulator.  They  found  that  dive  pitch  error  was  reduced  as 
scene  detail  increased.  In  addition,  augmented  feedback  aided  inexperienced  pilots  with  dive 
pitch  control  and  more  experienced  pilots  with  longitudinal  bombing  error. 

Lintem  et  al.  (1989)  examined  the  influence  of  scene  detail,  field  of  view,  and  number  of 
simulator  trials  on  performance  in  an  air-to-ground  mission  (i.e.,  radial  bombing  accuracy).  As 
with  Westra  et  al  (1986),  scene  detail  and  field  of  view  had  no  significant  effect  on  performance. 
However,  bombing  accuracy  improved  as  a  result  of  simulator  training.  Lintem  et  al.  noted  that 
no  additional  improvement  in  bombing  accuracy  was  achieved  after  24  simulator  sorties. 

Instrument,  and  Flight,  Control.  Simulators  are  often  used  for  training  instrument  and 
contact  flight  in  beginning  training.  Pfeiffer  et  al.  (1991)  evaluated  transfer  of  simulated 
instrument  training  to  instrument  and  contact  flight  for  a  limited  set  of  maneuvers  involving  turn 
and  speed  change.  Participants  completed  eight  simulator  flights  followed  by  two  aircraft  flights. 
Simulator  and  aircraft  performance  measures  consisted  of  deviations  from  assigned  airspeed, 
attitude,  and  heading.  Results  indicated  positive  transfer  from  the  simulator  to  the  aircraft.  A  test 
of  the  validity  of  simulator  performance  for  predicting  actual  flight  performance  showed  mean 
performance  in  the  simulator  to  be  strongly  related  to  both  instrument  (r  =  .98)  and  contact 
(r  =  .95)  flight. 

Goettl  (1995)  used  a  backward  quasi-transfer  technique  to  determine  transfer  effects  of 
part-task  training  to  complex  flight  control  tasks.  Participants  were  assigned  to  one  of  three 
training  groups.  The  first  group  received  whole-task  training,  the  second  received  part-task 
training  concentrating  on  the  critical  task  components,  and  the  third  group  received  part-task 
training  on  the  noncritical  task  components.  Final  performance  was  measured  in  the  simulator 
using  four  slalom  courses  (i.e.,  two  easy  and  two  difficult  courses),  where  participants  had  to  fly 
through  “gates  in  the  sky.”  The  groups  who  received  part-task  training  on  the  critical  components 
and  the  group  who  received  whole-task  training  did  not  differ  in  performance,  while  those  who 
received  non-critical  part-task  training  performed  significantly  worse. 
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Both  studies  suggest  that  simulators  provide  an  effective  means  to  train  instrument 
procedures  and  flight  control.  The  results  suggest  that  in  order  to  produce  transfer  to  the  aircraft 
it  may  be  necessary  to  train  only  the  critical  components  of  the  task  rather  than  the  whole  task. 
This  has  implications  for  simulator  design  and  the  development  of  simulator  training  syllabi. 


PROBLEM  AREAS 

Evaluation  of  the  transfer  of  training  literature  is  difficult  for  many  reasons.  Typically, 
researchers  fail  to  report  sufficient  detail  regarding  research  methods  (e.g.,  assignment  of 
participants  to  control  and  experimental  groups,  reasons  for  loss  of  participants,  methods  used  to 
estimate  interrater  agreement),  training  characteristics  (e.g.,  general  training  features,  instructor 
variables,  student  variables,  training  program),  and  simulator  fidelity  characteristics  (Hays  et  al., 
1990).  These  problems  have  not  been  resolved  in  the  last  ten  years.  For  example,  Joma,  Van 
Kleef,  and  de  Boer  (1992)  noted  that  the  mission  requirements  often  are  not  well  understood, 
making  it  difficult  to  design  an  adequate  training  syllabus  for  the  simulator  and  identify  a 
measure  of  performance.  According  to  Taylor  et  al.  (1993),  the  development  and  implementation 
of  design  principles  to  facilitate  maximization  of  training  transfer  and  cost  effectiveness  have  not 
been  well  established. 

In  addition  to  these  methodological  concerns,  this  literature  suffers  from  the  lack  of  true 
simulator-to-aircraft  transfer  studies  involving  complex  pilot  skills.  This  may  be  due  to  a  number 
of  reasons  including  inadequate  simulator  design,  cost,  and  availability  and  access  to  simulators 
in  operational  flying  units. 


FUTURE  DIRECTIONS 

Recent  developments  in  technology  have  led  to  advances  in  simulator  design.  For 
example,  faster,  more  powerful  computers  make  it  possible  to  link  several  simulators  together  to 
create  a  virtual  multiship  environment.  Pilots  are  now  able  to  fly  four-ship  sorties  against 
multiple  adversaries  in  near  real  time.  In  addition,  the  simulator  visual  systems  have  improved  in 
that  more  realistic  terrain  mapping  is  now  possible. 

Given  the  current  status  of  simulator  design,  it  is  now  possible  to  investigate  a  wide  range 
of  flying  skills  in  both  novice  and  experienced  pilots.  Future  studies  will  be  able  to  address  issues 
such  as  complex  skill  acquisition  and  group  performance  involving  multiship  scenarios. 
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